It is common to produce consumer colour sensors by introducing an alternating pattern of colour filters onto the array of individual sensor elements. This is often referred to as a colour mosaic and a commonly used variant is the RGB Bayer pattern, which has alternating rows of green/red and blue/green pixels (thus having twice as many green pixels as red and blue) on a regular grid. The goal of colour plane reconstruction (also called de-mosaicing) is to generate a full RGB image from an image captured with such a sensor.
No single colour plane has the full image resolution. In terms of total pixel count, with the Bayer pattern, green has halt the resolution and red and blue each have one quarter. Because each colour plane is sampled at different points of the image, aliasing effects within the colour planes are offset with respect to each other. These effects (colour aliasing artefacts) can appear as conspicuous patterns of false colour in a reconstructed image, especially when the image contains high frequency features such as thin lines and sharp edges.
Lines and edges that are angled away from the horizontal and vertical axes of the filter pattern may cause patterns of rapidly alternating false colours, referred to herein as short range colour aliasing artefacts. For example, if a thin pale line at 45 degrees to the axes of the array lands mostly on a diagonal containing red and blue alternating filters, the reconstructed image may contain false red and blue colours along the line. Lines and edges that lie at angles closer to the horizontal or vertical axes of the filter pattern may cause patterns of much more slowly changing false colours, referred to herein as long range colour aliasing artefacts. For example, if a nearly horizontal thin line begins on a row of the sensor having red and green filters and ends on a row having green and blue filters, the reconstructed image may show the line starting orange and ending blue/cyan.
A number of approaches have been proposed to deal with these problems. Reduced colour aliasing effects are obtained using a method described in U.S. Pat. No. 5,629,734. Here the green channel is processed first using an adaptive interpolation scheme that depends upon both horizontal and vertical intensity local gradient measures (computed from green pixels alone and/or complementary red or blue pixels also). The green signal is interpolated either along the row or column depending which direction has the smallest gradient (if the gradients are equal the average of the row and column interpolation is preferred). The interpolation can involve a correction term derived from the neighbouring red or blue pixels. This results in a complete green image that is free from local patterning whose content is then used to interpolate both the red and blue versions. Thus, image pixels for each colour value in the original mosaic image are directly interpolated to fill in missing pixel locations in such a way as to reduce colour aliasing effects. The bi-directional adaptive techniques used do, however, still require significant processing powering, and zippering effects are not eliminated or reduced as much as may be desirable. This approach does not deal with the problem of long range colour aliasing.
Another approach that does not rely on computation is to optically blur the image before it lands on the image sensor. Various simple demosaicing schemes can then be employed to reconstruct the full RGB image without risk of introducing false colours. Blurring the image avoids the high frequencies that give rise to the artifacts, but the image must be blurred enough to remove features much smaller in size than a 2×2 group of pixels (i.e. smaller than a group of pixels containing at least one of each of the colour filters). This method is employed in almost all digital still cameras. Typically they have a birefringent anti-aliasing filter immediately in front of the sensor to blur the image in a controlled manner. Such cameras avoid long and short range colour aliasing but at the cost of image resolution.
It is an object of the current invention to provide a more convenient and economical apparatus and method for reconstructing a full colour image from image mosaic data, with reduced or eliminated colour aliasing effects.